Extreme Acceleration of Graph Neural Network-based Prediction Models for Quantum Chemistry

Molecular property calculations are the bedrock of chemical physics. High-fidelity \textit{ab initio} modeling techniques for computing the molecular properties can be prohibitively expensive, and motivate the development of machine-learning models that make the same predictions more efficiently. Training graph neural networks over large molecular databases introduces unique computational challenges such as the need to process millions of small graphs with variable size and support communication patterns that are distinct from learning over large graphs such as social networks. This paper demonstrates a novel hardware-software co-design approach to scale up the training of graph neural networks for molecular property prediction. We introduce an algorithm to coalesce the batches of molecular graphs into fixed size packs to eliminate redundant computation and memory associated with alternative padding techniques and improve throughput via minimizing communication. We demonstrate the effectiveness of our co-design approach by providing an implementation of a well-established molecular property prediction model on the Graphcore Intelligence Processing Units (IPU). We evaluate the training performance on multiple molecular graph databases with varying degrees of graph counts, sizes and sparsity. We demonstrate that such a co-design approach can reduce the training time of such molecular property prediction models from days to less than two hours, opening new possibilities for AI-driven scientific discovery

Reducing Down(stream)time: Pretraining Molecular GNNs using Heterogeneous AI Accelerators

The demonstrated success of transfer learning has popularized approaches that involve pretraining models from massive data sources and subsequent finetuning towards a specific task. While such approaches have become the norm in fields such as natural language processing, implementation and evaluation of transfer learning approaches for chemistry are in the early stages. In this work, we demonstrate finetuning for downstream tasks on a graph neural network (GNN) trained over a molecular database containing 2.7 million water clusters. The use of Graphcore IPUs as an AI accelerator for training molecular GNNs reduces training time from a reported 2.7 days on 0.5M clusters to 1.2 hours on 2.7M clusters. Finetuning the pretrained model for downstream tasks of molecular dynamics and transfer to a different potential energy surface took only 8.3 hours and 28 minutes, respectively, on a single GPU.Comment: Machine Learning and the Physical Sciences Workshop at the 36th conference on Neural Information Processing Systems (NeurIPS

Unsupervised segmentation of irradiation\unicode{x2010}induced order\unicode{x2010}disorder phase transitions in electron microscopy

We present a method for the unsupervised segmentation of electron microscopy images, which are powerful descriptors of materials and chemical systems. Images are oversegmented into overlapping chips, and similarity graphs are generated from embeddings extracted from a domain\unicode{x2010}pretrained convolutional neural network (CNN). The Louvain method for community detection is then applied to perform segmentation. The graph representation provides an intuitive way of presenting the relationship between chips and communities. We demonstrate our method to track irradiation\unicode{x2010}induced amorphous fronts in thin films used for catalysis and electronics. This method has potential for "on\unicode{x2010}the\unicode{x2010}fly" segmentation to guide emerging automated electron microscopes.Comment: 7 pages, 3 figures. Accepted to Machine Learning and the Physical Sciences Workshop, NeurIPS 202

AI3SD Video: Preserving Structural Motifs in Machine-Learning Approaches to Modeling Water Clusters

Chemical structures are naturally viewed as collections of atoms connected through bonds, and graph theory provides a natural tool for capturing that intuition in a concrete mathematical fashion. Over the past several years, graph neural networks have become increasingly popular for modeling chemical systems. To build on this work, the multi-laboratory ExaLearn project, part of the DOE Exascale Computing Project, is developing novel capabilities that combine state-of-the-art machine-learning techniques with high-performance computing to enable the rapid exploration of chemical space on exascale-class systems. Water clusters offer an interesting use case for the development of machine-learning approaches that preserve intermolecular interactions and structural motifs. We apply a dataset of ~5 million hydrogen-bonded water clusters that display interesting long-range structural patterns to explore unique challenges in property prediction and molecular generation.[1] J. A. Bilbrey, J. P. Heindel, M. Schram, P. Bandyopadyay, S. S. Xantheas, S. Choudhury. “A look inside the black box: Using graph-theoretical descriptors to interpret a Continuous-Filter Convolutional Neural Network (CF-CNN) trained on the global and local minimum energy structures of neutral water clusters,” J. Chem. Phys., 2020, 153, 024302.[2] S. Choudhury, J. A. Bilbrey, L. Ward, S. S. Xantheas, I. Foster, J. P. Heindel, B. Blaiszik, M. E. Schwarting. “HydroNet: Benchmark Tasks for Preserving Long-range Interactions and Structural Motifs in Predictive and Generative Models for Molecular Data,” Machine Learning and the Physical Sciences workshop at NeurIPS, 2020

An open database of computed bulk ternary transition metal dichalcogenides

Abstract We present a dataset of structural relaxations of bulk ternary transition metal dichalcogenides (TMDs) computed via plane-wave density functional theory (DFT). We examined combinations of up to two chalcogenides with seven transition metals from groups 4–6 in octahedral (1T) or trigonal prismatic (2H) coordination. The full dataset consists of 672 unique stoichiometries, with a total of 50,337 individual configurations generated during structural relaxation. Our motivations for building this dataset are (1) to develop a training set for the generation of machine and deep learning models and (2) to obtain structural minima over a range of stoichiometries to support future electronic analyses. We provide the dataset as individual VASP xml files as well as all configurations encountered during relaxations collated into an ASE database with the corresponding total energy and atomic forces. In this report, we discuss the dataset in more detail and highlight interesting structural and electronic features of the relaxed structures

Rapid Electrochemical Reduction of Ni(II) Generates Reactive Monolayers for Conjugated Polymer Brushes in One Step

This article reports the development of a robust, one-step electrochemical technique to generate surface-bound conjugated polymers. The electrochemical reduction of arene diazonium salts at the surface of a gold electrode is used to generate tethered bromobenzene monolayers quickly. The oxidative addition of reactive Ni(0) across the aryl halide bond is achieved in situ through a concerted electrochemical reduction of Ni­(dppp)­Cl₂. This technique limits the diffusion of Ni(0) species away from the surface and overcomes the need for solution deposition techniques which often require multiple steps that result in a loss of surface coverage. With this electrochemical technique, the formation of the reactive monolayer resulted in a surface coverage of 1.29 × 10¹⁴ molecules/cm², which is a 6-fold increase over previously reported results using solution deposition techniques

On the Role of Disproportionation Energy in Kumada Catalyst-Transfer Polycondensation

Kumada catalyst-transfer polycondensation (KCTP) is an effective method for the controlled polymerization of conjugated polymers. Nevertheless, side reactions leading to early termination and unwanted chain coupling cause deviations from the target molecular weight, along with increasing polydispersity and end group variation. The departure from the KCTP cycle stems from a disproportionation reaction that leads to experimentally observed side products. The disproportionation energies for a series of nickel-based initiators containing bidentate phosphino attendant ligands were computed using density functional theory at the B3LYP/DZP level. The initiator was found to be less favorable toward disproportionation by 0.5 kcal mol^–1 when ligated by 1,3-bis­(diphenylphosphino)­propane (dppp) rather than 1,2-bis­(diphenylphosphino)­ethane (dppe). Trends in disproportionation energy (<i>E</i>_disp) with a variety of bidentate phosphine ligands match experimental observations of decreased polymerization control. Theoretical <i>E</i>_disp values can thus be used to predict the likelihood of disproportionation in cross-coupling reactions and, therefore, aid in catalyst design